How confident are you in your study?

‘How many participants do I need for my study?’ During many years of providing statistical support to clinical researchers, this has been one of the most frequently asked questions. It is posed by medical students preparing dissertation projects through to seasoned professors preparing major grant proposals. The repetition of this question can be frustrating, but does at least indicate that the researchers are aware – if only vaguely – of the importance of calculating in advance the number of participants needed for their study. But why is this so important? Put simply, failure to consider a study’s optimum sample size may render the study unethical. For how can it be ethical to ask patients to undergo the rigours and possible risks of a clinical trial if there is no evidence that the sample size is sufficient to have an acceptable chance of achieving a clinically useful result? This is why many medical research ethical committees now routinely ask for a power (sample size) calculation. (If yours does not, it should!). The mathematics involved is not particularly onerous and there are inexpensive software packages available that can help (e.g. nQuery, Power and Precision). If a study sample is too small to detect a clinically important effect, future patients may be denied the benefits of an effective treatment until a more definitive (powerful) study is completed. Studies can also be too large. Some years ago, a researcher provided his Research Ethics Committee with a power calculation showing that a sample of 160 patients was the optimum number for his study objective. However, he proceeded to recruit 240 patients. On terminating the study, the difference between the treatments was found to be statistically highly significant (p<0.001). Had he recruited 160 patients as originally planned, the difference would still have been significant (p<0.01). Therefore, 80 patients had been through the study needlessly and 40 of them had received an inferior treatment. The sample size for a comparative study is determined in order to provide an acceptably high chance (power) of detecting a clinically significant difference between the study groups, if one exists. Conventionally, power levels of 80 to 90% are considered desirable. For example, a study to investigate the differences in verbal IQ between children with and without ADHD may choose a measure of verbal IQ that has an inherent standard deviation of 20 units and that a difference of 12 units would be considered clinically significant. A simple calculation indicates that a study of 45 ADHD and 45 control children would have 80% power (i.e. an 80% chance of detecting a real difference of 12 units or greater). Increasing the sample size to 60 participants per group would raise the power to 90%. However, it is common for groups as small as 15 or even 12 children to be used in studies of this type. Such samples provide about 33% power which means that if there is a real difference in verbal IQ of 12 or more units between the two groups, there is only about a 1 in 3 chance of detecting it. E diorial